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DETAILED ACTION 

1 . This action is responsive to communications: Request for RCE and Amendment file on 
2/22/05. 

2. Claim 1 1 is cancelled. Claims 1-10 and 12-28 are pending. Claims 1,17, and 21 are 
independent claims. 

Continued Examination Under 37 CFR LI 14 

3. A request for continued examination under 37 CFR 1.114, including the fee set forth in 
37 CFR 1.17(e), was filed in this application after final rejection. Since this application is 
eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) 
has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 
37 CFR 1.1 14. Applicant's submission filed on 2/22/05 has been entered. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 1-4, 8, 12-13, 16-21, 23-25, and 27-28 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Mukhopadhyay et al, ("Passive Capture and Structuring of 
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Lectures", 10/1999, ACM Multimedia '99, pages 1-11 (submitted in the IDS filed on 

9/14/00)) in view of Mitchell et al. (USPN 5,963,966 - issued on 10/1999). 

Regarding independent claim 1 and (dependent claims 13, 23, 25, and 27), Mukhopadhyay 

discloses: 

A method for linking a document to a video (Mukhopadhyay on page 2, right column teaches 
slides are synchronized with the video; on page 3, left column teaches slides are matched to the 
video; and on page 10, left column teaches inserting digitized output of scan converters when 
demonstrations are given), comprising the steps of: 

document identifier (Mukhopadhyay on page 5, see figure 7 teaches each slide image are 
set with a value to match with each video frame); 
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obtaining a video file having a plurality of video frames (Mukhopadhyay on page 5 
teaches video sequence of a plurality of segments); 

transforming the plurality of video frames into plurality of respective video frame 
identifiers (Mukhopadhyay on page 5 teaches matching a representative frame from each 
segment in T to one of the slides in S; a video sequence comprises a set of time values T ); 

comparing the document identifier with the plurality of video frame identifiers 
(Mukhopadhyay on page 5 teaches computing a video sequence of v(t) segments or frames to 
match against every slide image S (see figure 7)); and 

linking the document to a first video frame in the plurality of video frames 
(Mukhopadhyay on page 5, see figure 7 teaches matching slides in each frame to synchronize 
video data to the slides). 

However, Mukhopadhyay does not explicitly disclose "obtaining a scanned document 
having margins", "removing margins from scanned documents", "scaling the scanned 
document", "removing the least significant information", and "transforming the scanned 
document". 

Mitchell discloses "obtaining a scanned document having margins", "removing margins 
from scanned documents" and "scaling the scanned document" on col. 8, lines 36-43 and col. 9, 
lines 18-19 teaches document pages are scanned into images; wherein the images are cropped to 
remove the borders and rescaled; "transforming the scanned document" on col. 6, lines 36-40 
teaches translating from paper to electronic form such as SGML, HTML, and text. 

Further, Mitchell discloses the new amended feature "removing the least significant 
information from the scanned document", on col. 8, lines 40-43 teaches since the sample 
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document pages contain large white borders that do not contribute any information, the images 

were cropped to remove the borders. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 2, Mitchell discloses scanning a document to create an image on 
col. 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 3, Mitchell discloses cropping the image from the scanned 
document to remove the borders on col. 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
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paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 4, Mukhopadhyay discloses a reduced image of the slide displayed 
on the Lecture Browser user interface to fit the corresponding video within the user interface (see 
figure 2). 

Regarding dependent claims 8 and 19, Mukhopadhyay discloses using a camera to record the 
presentation on page 2. 

Regarding dependent claims 12 and 28, Mukhopadhyay discloses comparing color histograms 
of successive frames with the slides on page 5, right column - page 6, left column. 
Regarding dependent claim 16, Mitchell discloses document images contain hyperlinks on col. 
8, lines 60-67. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding independent claims 17 and 21, Mukhopadhyay discloses "link between a document 
file and a segment of a video file responsive to the comparison of a transformed document and 
transformed video frame", on page 5, see figure 7 teaches matching slides in each frame to 
synchronize video data to the slides). 
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However, Mukhopadhyay does not explicitly disclose "removing the least significant 
information from the scanned document" and "scanned document". 

Mitchell discloses "a scanned document" on col. 8, lines 36-43 and col. 9, lines 18-19 
teaches document pages are scanned into images. 

Further, Mitchell discloses the new amended feature "removing the least significant 
information from the scanned document", on col. 8, lines 40-43 teaches since the sample 
document pages contain large white borders that do not contribute any information, the images 
were cropped to remove the borders. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 18, Mitchell discloses "a scanner to create scanned document from 
a physical document", on col. 8, lines 36-43 and col. 9, lines 18-19 teaches document pages are 
scanned into images using OCR. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
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paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

Regarding dependent claim 20, Mukhopadhyay discloses a Lecture Browser user interface for 
viewing slides and video on page 2, see figure 2. 

Regarding dependent claim 24, Mitchell discloses removing borders from the scanned 
document on col. 8, lines 36-43. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay, in order to provide a low-cost and high performance solution for converting 
paper into a form that can be accessed through the network such as the Internet or a media such a 
disk or CD-ROM. 

6. Claims 5-7, 9-10, and 22 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Mukhopadhyay and Mitchell, as applied to claims 1 and 21 above, further in view of 
Kumar (USPN 5,835,129 - issued on 11/1998). 

Regarding dependent claims 5-6, 9-10, and 22, Mukhopadhyay and Mitchell disclose the 
invention substantially as claimed as described supra. However, Mukhopadhyay and Mitchell 
do not explicitly disclose "orthonormal transform" and "discrete cosine transform". 

Kumar discloses transformation of the image frame and the process of dividing image 
frames into blocks using DCT (discrete cosine transform) defined by the orthonormal basis on 
col. 8, line 60 - col. 9, line 3. 
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It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Kumar into Mukhopadhyay and Mitchell to provide the 
transformation of an image using DCT defined by the orthonormal basis, as taught by Kumar, 
incorporated into the images of Mukhopadhyay and Mitchell, in order to facilitate the division of 
image frames into blocks. 

Regarding dependent claim 7, Mitchell discloses "removing the least significant information 
from the scanned document", on col. 8, lines 40-43 teaches since the sample document pages 
contain large white borders that do not contribute any information, the images were cropped to 
remove the borders. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Mitchell into Mukhopadhyay to provide a way to scan 
document pages into images, as taught by Mitchell, incorporated into the slide image of 
Mukhopadhyay and image of Kumar, in order to provide a low-cost and high performance 
solution for converting paper into a form that can be accessed through the network such as the 
Internet or a media such a disk or CD-ROM. 

7. Claims 14-15 and 26 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mukhopadhyay and Mitchell, as applied to claims 1 and 21 above, further in view of 
Bloomberg (USPN 5,181,255 - issued on 01/1993). 

Regarding dependent claims 14-15 and 26, Mukhopadhyay and Mitchell disclose the invention 
substantially as claimed as described supra. However, Mukhopadhyay and Mitchell do not 
explicitly disclose "extracting handwritten annotations from scanned documents". 
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Bloomberg discloses extracting handwritten annotations from a scanned image on col. 1, 
lines 24-33, see Abstract. 

It would have been obvious to a person of ordinary skill in the art at the time the 
invention was made to have modified Bloomberg into Mukhopadhyay and Mitchell to provide a 
way to extract handwritten annotations from a scanned image, as taught by Bloomberg, 
incorporated into the images of Mukhopadhyay and Mitchell, in order to provide an improved 
system to identify and separate printed text and handwritten annotations from an image. 

Response to Arguments 
8. Applicant's arguments filed 2/22/05 have been fully considered but they are not 
persuasive. 

The Applicant is advised to cancel dependent claim 7, because independent claims 1,17, 
and 21 contains this limitation. 

Regarding Applicant's remarks on pages 9-13: 

Applicant argues that Mukhopadhyay does not teach "linking a document to a video", 
however, Mukhopadhyay on page 2, right column, page 3, left column, and page 5, see Figure 7: 
teaches slides (document) are matched and synchronized with the video, in other words, the 
slides can be "connected" and related to the video, frame- by-frame. 

Applicant argues that Mukhopadhyay does not teach "transforming the scanned 
document into a scanned document identifier". 
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Mukhopadhyay discloses each slide image is set with a value or identifier to facilitate the 
matching of each slide to each video frame (on page 5, see figure 7). Even though, 
Mukhopadhyay does not disclose the "transformation of the scanned document", Mitchell 
discloses the translation of paper to an electronic document (col. 6, lines 36-40). Furthermore, 
Mitchell discloses when the electronic document such as HTML document is created from 
scanning the paper document, an HTML address (identifier) is used to view the representative 
document of the scanned HTML document (see col. 7, lines 7-14). 

Applicant argues that Mukhopadhyay does not teach "obtaining a video file having a 
plurality of video frames", however, Mukhopadhyay does disclose "a video file having video 
frames" on page 5 teaches ". . .every frame of the overview video and matching each frame 
against all slide images" and Figure 7 shows a video sequence of a plurality of segments to 
facilitate the matching to slides. Furthermore, Mukhopadhyay on page 6 teaches methods for 
computing which slide corresponds to the video frame (See Figure 8). 

Applicant argues that Mukhopadhyay, Mitchell, and Kumar do not teach "orthonormal 
transforms which are performed at once on the entire scanned document". 

Kumar discloses the orthonormal transformation on an image frame on col. 8, line 60- 
col. 9, line 3, however, Kumar's invention can also transform still images, electronic white 
boards, slides (see col. 4, lines 30-38), in other words, Kumar is not limited to only image 
frames. 
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Applicant argues that Mukhopadhyay, Mitchell, and Kumar do not teach "removing the 
least significant information", however, Mitchell does teach removing the least significant 
information, on col. 8, lines 36-43 teaches since the sample document pages contain large white 
borders that do not contribute any information, the images were cropped to remove the borders. 



examiner should be directed to Almari Yuan whose telephone number is 571-272-4104. The 
examiner can normally be reached on Mondays - Fridays (8:30am - 5:00pm). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph Feild, can be reached on 571-272-4090. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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